Add a (WIP) simple test vector parsing crate #60
Conversation
The idea here is to provide a crate that people can depend on to parse the golden test vectors. It's very unfinished, and ultimately should just be a thin wrapper over a canonical ledger state crate provided by pallas or similar. Many things are marked as AnyCbor because I didn't track down what types they should be. I'd also like to provide a feature flag where the test vectors themselves can be `include_bytes!`'d, to save people from having to download the test vectors directly themselves.
|
|
woops haha thanks @yHSJ |
|
Hey, very cool. Could you please add (to the PR description), a "How to Use" and "How to Test" section? I could probably figure it out, but I like to use the "Don't Make Me Think" rule on most things, which ultimately serves as historical doc as well. Doesn't have to be much - even just a couple of shell commands. I realized after reading the test that it just confirms it can parse the data, but does not validation. I think a nice TODO for the future is to check some values. Test...How do I use this crate? With a GH reference???? |
|
Okay, so I do see nice README descriptions which could be enough, but I have an even dumber question... Why is this rust code in the blueprints repo? I understand that Blueprints is supposed to be implementation agnostic, but seems like we're taking over the src/ledger directory with Rust. Might be fine, but just asking the question. |
|
This looks really great for golden test vectors! Thanks for knocking this out and the nice example by way of the test. How would this relate to parsing a snapshot file? Would this be usable for reading the LedgerState component? Presuming we add a full snapshot parsing capability would you imagine putting that in this repo as well? |
|
A test vector is just composed of two As for why this is in the cardano blueprint, because that's where the test vectors live and get updated long term; that's the real artifact that is being delivered, and people are free to consume it however they wish. But, we provide the rust crate as a convenience / documentation for how the files are laid out, which then lets anyone write their own parser. If there's enough demand, I wouldn't be opposed to having alternative implementations of the parser in other languages, but I also wouldn't be opposed to them living elsewhere. The point of cardano-blueprint isn't to provide an authoritative crate for reading the test vectors, and you're SOL if you're in another language; the point is to make the test vectors available for consumption, and as a secondary convenience we provide a parser you can work from. I'll let @ch1bo comment on whether he feels differently about organization, etc. |
Generally, the blueprints are meant to be a shared repository that can be used as a reference to implement Cardano, and related tooling. The piece that is the most valuable here is the golden test vectors. The Rust just provides an example of parsing the raw bytes, which is very useful. Perhaps it's worth also providing some prose in the blueprints themselves (not just the Regarding the actual location of said logic, that is a great question. It probably makes sense to have a separate directory under the |
|
@Quantumplation in the test vectors, we have transactions and I assume there are transactions in the snapshot somewhere. So possibly a snapshot parser could wrap the NodeState and transactions. Maybe instead of tweaks, it's just a wrapper and a few more types. Thanks for the explanation. |
|
No, there are no transactions in the snapshot 😅 A golden test vector is an initial snapshot, followed by a sequence of transactions to apply, and a final snapshot to compare your state to. The snapshot itself is just the UTxOs, account state, reward pots, etc. |
We've been working to produce a set of golden test vectors, generated from the tests in the haskell node.
Working with these is a little awkward however; eventually these will just be slim wrappers over the canonical ledger state being worked on tweag, but we're quite a ways from that, so this PR introduces a crate that makes loading and working with these in Rust more convenient.
It's very much a WIP, but figured I would get this pushed up so people can start consuming and iterating on it.
Many things are marked as AnyCbor because I didn't track down what types they should be.
I'd also like to provide a feature flag where the test vectors themselves can be
include_bytes!'d, to save people from having to download the test vectors directly themselves.